Reclaiming one’s bandwidth: Dynamic filtering of traffic based on packet payload content

Dynamic filtering of IP traffic based on contents of packet payloads provides a means for monitoring and controlling a number of Internet services which prove difficult to manage by traditional means. Constituents of the portion of Internet traffic not attributable to traditional services can be identified and quantified, even if applications transfer data on a highly dynamic range of ports. An initial implementation of a dynamic filtering system under FreeBSD is also evaluated and strengths and weaknesses identified. 1. The need for dynamic filtering In recent months applications have been developed to enable users to easily share a variety of file type easily over the Internet. Many of these build on the groundwork for sharing of mp3 media files pioneered by Napster[10]. Soon after its release, Napster became immensely popular and, due to the sheer volume of users making use of the application brought many institutions Internet connections to a standstill. The nature of mp3 files are that they are large, with a typical file consuming between three and four megabytes of disk space, depending on the encoding options chosen when the file was initially compressed. Following on from this popularity, other authors produced similar software, such as Imesh[4], cuteMX[2] and Gnutella[3] that was extended to potentially be able to share any file type. Of particular note is the development of Gnutella which utilises a protocol designed to be difficult to filter [6]. Many of these applications are also able to operate if one party in the peer−to−peer connection is behind a firewall. [2,3,4,6] Not all these applications work on the basis of connecting to some form of central server which is used for collating files being offered by connected clients. Napster uses a variety of ports and a number of servers which are tried when a connection is made. Thus, in order to block such traffic, firewall administrators would have to either block all outgoing connections, or spend time keeping up to date with server lists and block access to those. This task could become quite time consuming, yet indiscriminately blocking all possible ports that could be used by such applications would soon result in legitimate services being disrupted. Gnutella as feature if its design, allows any node on the network to act as an uplink server for new new nodes joining the network. File sharing utilities are not the only cause for concern, monitoring of other network traffic of dubious intent may also be of use. Many ’warez’ servers (servers containing illegal copies of software for download) make use of Internet Standard protocols such as FTP and HTTP, but run these services on non standard ports. These are usually undesirable either because of the content contained, or the lack of accounting information (’audit trail’) for this traffic, which is the common case where a proxy server has been deployed. These same methods can be used for monitoring connections to an external proxy server which would allow users to bypass any local proxy restrictions. Similarly with external SMTP, POP, IMAP and News servers. This could be regarded as a security measure −− especially if the organisation has intellectual property which it wishes to prevent being disseminated via e−mail or other electronic means. A solution to the monitoring and filtering of these traffic types can be implemented with methods commonly used by Intrusion Detection Systems (IDS), where traffic entering a LAN is scanned for various signatures indicating a likely hostile probe or attack against machines on the network. An open source example of such an IDS is Snort[13,14] written by Martin Roesch and which comes with a rich library of traffic signatures. Traffic is inspected at the packet level, and packets with a payload matching certain predefined rules is reported as suspicious. Optionally the system can generate an appropriate firewall rule for the specific connection matching a rule such as just adding an accounting entry, constricting traffic bandwith or denying the connection. Use of such a system allows legitimate traffic running on ’suspect ports’ through with no interference. 1 MPEG Layer 3 Audio Compression